Automatic Face Recognition from Video

The objective of this work is to automatically recognize faces from video sequences in a realistic, unconstrained setup in which illumination conditions are extreme and greatly changing, viewpoint and user motion pattern have a wide variability, and video input is of low quality. At the centre of focus are face appearance manifolds: this thesis presents a significant advance of their understanding and application in the sphere of face recognition. The two main contributions are the Generic Shape-Illumination Manifold recognition algorithm and the Anisotropic Manifold Space clustering. The Generic Shape-Illumination Manifold algorithm shows how video sequences of unconstrained head motion can be reliably compared in the presence of greatly changing imaging conditions. Our approach consists of combining a priori domain-specific knowledge in the form of a photometric model of image formation, with a statistical model of generic face appearance variation. One of the key ideas is the reillumination algorithm which takes two sequences of faces and produces a third, synthetic one, that contains the same poses as the first in the illumination of the second. The Anisotropic Manifold Space clustering algorithm is proposed to automatically determine the cast of a feature-length film, without any dataset-specific training information. The method is based on modelling coherence of dissimilarities between appearance manifolds: it is shown how interand intra-personal similarities can be exploited by mapping each manifold into a single point in the manifold space. This concept allows for a useful interpretation of classical clustering approaches, which highlights their limitations. A superior method is proposed that uses a mixture-based generative model to hierarchically grow class boundaries corresponding to different individuals. The Generic Shape-Illumination Manifold is evaluated on a large data corpus acquired in real-world conditions and its performance is shown to greatly exceed that of state-ofthe-art methods in the literature and the best performing commercial software. Empirical evaluation of the Anisotropic Manifold Space clustering on a popular situation comedy is also described with excellent preliminary results.